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ABSTRACT 
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1. Introduction. Notations. Statement of problem. 



Let (fi, A1,P) be a probability space with the expectation E, f2 = {tu}, (X, ^, /i) be 
a measurable space with sigma - finite non - trivial measure /x, (G, r) = B = {^} 
be arbitrary separable local compact topological space equipped by the ordinary Borelian 
sigma - field, = {/}, / = f{x,9) be a family of a strictly positive (mod /x) 
probabilistic densities: 

yeee ^ [ fix, 6) djj = 1, 

f^{Ueee{x: /(a;, 0) < 0}} = 0, 



We premise also the following natural condition of the identifying: 

v^i, 02 e e, ^1 ^ ^2 l^{x : fix, e,) ^ fix, 62)} > 0. 

Let further 9q e Qhe some fixed value of the parameter 9. We assume that ^ — ^ (a;) is 
a random variable (r.v) (or more generally random vector) taking the values in the space 
X with the density of distribution fix, 9q) relative the measure : 

P(eeA)= / fix,e,)d^i, AeM. (1.0) 

J A 

The statistical sense: the r.v. C, is the (statistical) observation (or observations) with 
density fix, 9q), where the value 6*0 is the true, but in general case unknown value of the 
parameter 9. 

We denote as usually by 9 the Maximum Likehhood Estimation (MLE) of the param- 
eter ^0 based on the observation ^ : 



or equally 



where the function 



9 = argmax/({,6'), 



9 — argmaxL(^, 9) — argmin(— L(^, 9)) (1.1) 
0ee eee 



L = L(e,^) =^ log [/(e,^)//(e,^o)] 

is called the contrast function, in contradiction to the function 9 — > /(^, ^) or ^ ^ 
log/(^,^^), which is called ordinary Likelihood function. 
Denote also 

L(°)=L(°)(e,e) = L(e,^) - BLi^,9)^Li^,9) - Eo Li^,9). 

In the case if 9 is not unique, we understand as 9 any but measurable value 9 satisfying 
the condition (1.1). 

Let now r = ri9) = ri9,9o), 9^0 be some (measurable) numerical non - negative 
risk, or deviation function, i.e. such that 

rid,eo) > 0, rid, do) =0 ^9^9o, 

not necessary to be distance, i.e. it can not satisfy the triangle inequality. We denote for 
arbitrary positive value v the probability of the deviation in the r(-, •) sense of 9 from the 
true value 9o : 

Wiv)'^^Piri9,9o)>v), (1.2) 

which is needed for the construction of confidence region for the unknown parameter 9o 
in the r(-, •) sense. 

Our goal of this paper is non - asymptotical estimation of the function 
W = Wiv) under minimal and natural conditions for sufficiently greatest values 



Offered here estimations are some generalizations of tiie paper [6]. See also [1], [3], [7], 
[8] , [22] and reference therein. 

The paper is organized as follows. In the next section we introduce the needed no- 
tations and conditions. In the section 3 we describe and recall auxiliary facts about 
exponential bounds for tail of maximum distribution of random fields. 

In the fourth section we will formulate and prove the main result of this paper. Further 
we consider as a particular case of smooth density function. 

In the six section we consider as an application the case of sample, i.e. the case when 
the observations ^ — {^i, i — 1,2, . . . ,n} are independent and identically distributed (i., 
i.d). 

In the last section 7 we consider some examples in order to illustrate the precision of 
the obtained results. 

Agreement: by the symbols C,Cj,C{i) we will denote some finite positive non - 
essential constants. 

2. Notations and conditions. Key Inequality. 

It is presumed that all introduced function there exist in some domains 
which is described below. 

U{v) =^ {9: eee, r{e, Bo) >v}, v> 0; (2.0) 

then 

W{v) = P{e G U{v)). (2.1) 

Let A{k), k = 1.2, ... he some numerical strictly increasing sequence, A{1) = 1. For 
instance, A{k) = k or A{k) = k^, A = const > or possible A{k) = C k^, k > k^. We 
introduce also the following measurable sets: 

Uk = Ukiv) ^{0: 9ee, r{9, 9o) e [ A{k) v, A{k + 1) ^; ] }. 
We observe: 

oo 

W{v)<J2Wk{v), (2.2) 

k=l 

where 

Wk{v) ^P{0e Ukiv)) = P {r{§, 0) e [ A{k) V, A{k + l)v]). 

Introduce also the KuUback - Leibler "distance", or relative entropy, or quasi - 
distance between the parameters 9 and as usually 

h{9) = h{9o, 9) = EL(e, 9)= I fix, 9o) log[f{x, 9)/f{x, 9o)] dii. (2.3) 

It is known that h{9) > and h{9) =0 ^ 9 ^ 9q. 
We denote also 



Y{v) = inf h{9) 



(2.4) 



and suppose Y{v) G (0, oo) for all sufficiently great values v. 

Further, define the following functions (some modifications of Hellinger's integral) 



and 



0(A) = sup [ log E exp [al(°) (^, 9) 



sup [logEexp(AL(^,^)) • exp(-A/i(^))] 

eeUiiv) 



sup 



[ f\x,d) f - \x,eo) dpi ■ exp(-A/i(^)) 



X(A) = sup 



logEexp[AL(°)(e,e) 



sup [logEexp(AL(^,^)) • exp(-A/i(^))] = 



sup 



j f\x,e) f - \x,9o) dpi ■ exp(-A/i(^)) 



(2.5) 



(2.5a) 



We suppose 0(A) < oo or correspondingly x(A) < oo for all values A in some interval 
of a view (0, Aq), Aq G (0, oo] : 



3Ao e (0, oo], VA e (0, Ao] =^ 0(A) < oo. 



(2.6) 



3Ao e (0, oo], VA e (0, Aq] x(A) < oo. 



(2.6a) 



Key inequality. 



Note that 



W{v)^P[ sup L{C,e)> sup L{^,e)]. 

\eeu{v) e^u{v) ) 

As long as L(,^, ^^o) = and 6q (and 6q ^ Ui) we conclude 

W{v) < P ( sup L{i,e) > ) = 
\e&{v) ) 

P [ sup (L(e,^) - EL(e,^)) > inf M^)) < 
\B&j{v) eeu{v) J 

P ( sup L°(e,^) > Y{v)] = P f sup [log(/(e,^)//(e,^o)) - h{e) ] > Y{v)] . (2.7) 



Therefore, we can use the known exponentially exact estimations of maximum random 
field distributions, see, for example, [1], [10], [12], [13], [14], [15], [18], [19], [20], [21] etc. 



3. Auxiliary facts. 



Let {fl,A4,P) be again the probability space, ^2 = {u}, T = {t} be arbitrary set, 
^(t), t E T he centered: E^(t) = separable random field (or process). For arbitrary 
subset S CT we denote for the values u > Uq, Uq — const e (0, oo) 

Q{S,u) =^P(supe(t) > u); Q{u) := Q{T,u). (3.1) 
tes 

Our (local) goal in this section is description an exponentially exact as 
w — > oo estimation for the probability Q{S,u), Q{u) in the so - called natural 
terms. 

Definitions and some important results about Esupjgy,^(t) in general, i.e. non - 
Gaussian case, i.e. when the random field ^{t) may be non - Gaussian, in the terms of 
majoring measures see, for example, in [1], [18], [19], [20], [21]. In the so-called entropy 
terms this problem was considered in [5], [12], [13], [14], [15] etc. 

In order to formulate our result, wc need to introduce some addition notations and 
conditions. Let = 0(A), A G [0, Aq), Aq = const G (0, oo] be some strictly convex taking 
non - negative values function, such that 0(0) = and 

A G [0, 0.5 Ao) =^ CiA' < 0(A) < CsA'; (3.2) 



lim 0(A)/A = oo. (3.3) 

A— >Ao 

Note that under the assumptions (2.5) or (2.5a) 0(-) G x(-) G $. 
We denote the set of all these function as$; $ = {0(-)}. 

Further we will choose T = U{v) or T — Uk{v) and 0(A) or correspondingly x(A) as 
it is defined as in (2.5) and (2.5a) . 

We say that the centered random variable (r.v) = 'C(^) belongs to the space B{(j)), if 
there exists some non - negative constant r > such that 

VA G [0, Ao) E exp(AO < exp[0(A r)], (3.4) 

(the concretization of right hand side Kramer's condition). 

The minimal value r satisfying (3.4) is called a S(0) norm of the variable ^, write 
1 1^1 1 or more detail, ||C||-B(0) : 

lieil = ||el|5(0) := mf{r, r > 0, VA Eexp(AO < exp(0(A r))}. (3.5) 

This spaces are very convenient for the investigation of the r.v. having a exponential 
decreasing right side tail of distribution, for instance, for investigation of the limit theorem, 
the exponential bounds of distribution for sums of random variables, non- asymptotical 
properties, problem of continuous of random fields, study of Central Limit Theorem in 
the Banach space etc.; see [13]. 

The space -8(0) with respect to the norm || ■ ||i^(0) and ordinary operations is a quasi 
- Banach space. This means by definition that: 



1. B{(f)) is complete relative the quasi - distance \ \i — rj 



2.||e||>0; lieiHO^e^O- 

the non - negativeness; 

3.||e + r/||<||e|| + ||r/||- 

the triangle inequality; 

4.Q; = const > =^ ||q;^|| = q;||^|| — 

the positive homogeneous. 

The B{(f)) is isomorphic to the subspace consisted on all the centered variables of quasi 

- Orlitz's space (Q, F, F),N(-) with N - right function 

N{u) = exp(0*(M)) - 1, (j)*{u) = sup{Xu - 0(A)). 

X 

The transform — > 0* is called Young - Fenchel or Legendre transform. The proof of 
considered assertion used the properties of saddle-point method and theorem of Fenchel 

- Moraux: 

(f)** = (f). 

Many facts about the B{4>) spaces are proved in [13], [14], p. 19 - 40: 
C e S(0) <^ = 0, and 3C = const > 0, 

Z(^,x) <ex.p(-(l)*(Cx)),x>0, (3.6) 
where Z{C^, x) denotes in this article the right hand tail of distribution of the r.v. ^ : 

Z(e,x) =^P(e>x), x>0, 

and this estimation is in general case asymptotically exact. 

More exactly, if Aq = oo, then the following implication holds: 

lim 0-^(logEexp(AO)/A = K e (0, oo) 

A— »oo 

if and only if 

lim i^*r\\^ogZ{^,x)\)/x=l/K. 



Here and further /~^(-) denotes the inverse function to the function / on the left - side 
half - line (C, oo). 

Let e We denote 



<t>n{X)^n<t>{X/y/^), <t>{X)= sup [n(/)(A/V^)] (3.7) 

n=l,2,... 



and a.na1np"niis1v 



Xn(A) = nx(A/Vri), x(A) = sup [n x(A/^/n)]. 



n=l,2,. 



(3.7a) 



This function obeys the following sense. If ^{i), i = 1, 2, ... be a sequence of centered, 
i., i.d. r.v., ^ = ^(1), belonging to the space B{4>) and having the unit norm in this space: 
1 1-8(0) = 1, then we have for the normed sum 

i=l 

E[exp(A ri{n))] < exp[0„(A)], 



sup Eexp(A r]{n)) < exp 0(A) 

n=l,2,... L 



(3.8) 



and following 



Z{ri{n),x) < exp [ - {(f)nT{x)] , 
the non - uniform estimation; 

sup Z{r]{n),x) < exp — (0)*(x) 



(3.9) 



the uniform estimation (Chernoffs estimations, see [4]). 

Using the property (3.2), we can show in addition to the classical theory of the great 
deviations that in the " mild" zone 

X = x{n) e (0, C y/n) ^ 



Z{7]{n),x) < C2 exp (-C; 



3 X 

(the non - uniform estimation). 
As an example: if in addition 

Z{^{i),x) < exp (— x^) , q — const > 1, x > 0, 
then for some constant C — C{q) e (0, 00) 

sup 

n=l,2,... 



(3.10) 



(3.11) 



and the last estimation is unimprovable at x >> 1. 
Now we prove a more general assertion. 

Lemma 3.1 Let {?7(i)}, i = 1, 2, . . . , n be a sequence of i., i.d., centered r.v. such 
that for some q — const > and for all positive values x 



Z{\ri{i)lx) <exp(-a;'^ R{x)) , 
where R{x) is continuous positive slowly varying as x — > 00 



(3.12) 



Vt > ^ lim R(tx)/R(x) ^ 1: 



is bounded from below in tiie positive semi - axis 



inf R(x) > 

x>0 



function. For instance: R{x) 



[\og{x + 3)Y, R{x) = [log{x + 3)Y ■ [log(log(x + 16))]', r, s = const, r > 0. 



Denote 



i=l 



We assert: 



supZ{\C{n)\,x) < min [exp (-Ci(g, i?) a;« R{x)) , exp (-C2{q, R) x'^) 
Notice that tlie lower bound, i.e. the inverse inequality 

sup Z{\(^{n)\,x) > min exp (— C3(g, i?) R{x)) , ex.p (—C4{q, R) x^) 

n '- V / 

in the case when 



Z{\r]{i)\,x) < e^p[-C5{q,R) x« R{x)] 



and 



Z{\r]{{)\,x) > ex.p[-CQ{q,R) x" R{x)] , < C5 > < oo 
is trivial. Namely, 

supZ{\an)\,x) > Z{\ri{l)\,x) > exp [-^(g, i?) x'^ R{x)] , 

n 

and on the other hand 

supZ{\C{n)\,x) > lim P(|CH| > x) ^ 



(3.13) 



(3.14) 



POD 

:/ (27r)- 

Jx 



(J ^ exp 



-|/7(2cT^) > exp -C^iq, R) x^ , a; > Cg > 0; 



here we used the CLT and denoted 

= a'^{q) = Var (r;(l)) e (0, 00). 

Proof (briefly) of the Lemma 3.1. 

The case > 1 is considered in [13], chapter 1, section (1.6); therefore we must consider 
only the case q e (0, 1). 



Further, without loss of generahty we can consider the case when the r.v. ^ and rj are 
independent and symmetrical distributed with densities correspondingly 



/^(x) = /(x) = Cg exp(-|x|« Ri\x\)), 

gr,{x) = g{x) = Cio exp {-K R{\x\)) , 

and r = ^ + r). Here X = const e (l,oo) (the case when K — 1 may be considered 
analogously) . 

Let us assume that x ^ oo, x > C. We have denoting by h{x) = hr{x) the density of 
distribution of the r.v. r : 

hix)r^Cu rexp[-K{x - y)m{x - y) - ym{y)\ dy ^ 
Jo 

Cux C exp [-x^ [Kil - tYRix (1 - t)) + R(t x)]] dt ~ 

Cux r exp R(x) S(t)] dt, 
Jo 

where 

S(t)^K(l - ty + t''. 

The function t — > S'(i), t e [0,1] achieves the minimal value K at the (critical) point 
t — and as i — > 0+ 

5(i) ^K + f^ + Oit). 

Note that in the case K = 1 there are two critical points: t — and i = 1. 
Further, we use the classical saddle - point method: at x — > oo, x > 1 we have: 

h(x) ~ Cnx / exp [-x^ i?(a;) (K + t«)] = 
Jo 

Ci2{q,R) X exp[-K x« R{x)] {x'^ R{x)y^''^ < 

Cisiq, R) exp [-K x'^ R(x)] . 
This completes the proof of the lemma 3.1. 

The function </>(■) may be introduced constructive, i.e. only by means of the values of 
the considered random field {^(t), t G T} by the formula 

(/.(A) = 0o(A) logsupEexp(Ae(t)), (3.15) 

if obviously the family of the centered r.v. {^(t), t € T} satisfies the uniform Kramers 
condition: 

3C e (0, oo), sup Z{^{t), x) < exp(-C x), x>0. (3.16) 
In this case, i.e. in the case the choice the function 0(-) by the formula (3.15), we will 



Note that if for some C = const G (0, oo) 



Q{T,u) <exp{-(f>*{Cu)), 

then the condition (2.6) is satisfied (the necessity of the condition (2.6)). 

M.Talagrand [18], [19], [20], [21], W.Bednorz [2], X. Fernique [5] etc. write instead our 
function exp (— </)*(x)) the function l/^(a;), where ^'(■) is some Youngs function and used 
as a rule a function \l/(a;) = exp(a;^/2) (the so - called subgaussian case). 

Without loss of generality we can and will suppose 

sup[\m iis(0)] = i, 

teT 

(this condition is satisfied automatically in the case of natural choosing of the function 
4> : 0(A) = 0o(A) ) and that the metric space {T,d) relatively the so-called natural 
distances (more exactly, semi - distances) 

d^{t,s)^d{t,s)'i^Um - amsicp), (3.17) 

and analogously (see the definition of a function %(•) further) 

dAt,s)'^^\m - as)\mx) (3.17a) 

is complete. 

Recall that the semi - distance p = p{t, s), s, t e T, for instance, d = d^{t, s), s,t & T 
is, by definition, a non - negative symmetrical numerical function, p{t, t) = 0, t & T, 
satisfying the triangle inequality, but the equality p{t, s) — does not means (in general 
case) that s — t. 

For example, if the random field ^{t) is centered and normed: 

supVar [e(t)] = l 
teT 

Gaussian field with a covariation function D{t, s) — E^(i) ^(s), then (f)o{X) — 0.5 A^, A e 
R, and d{t, s) — 

\m - e(«)l|5(0o) = VVar[e(t) - e(s)] = ^D(t,t) - 2D(t, s) + D(s, s). 

Let (T, p) be a compact metrical space. Wc us introduce as usually for any subset 
V, V (Z T the so-called entropy H{V, p, e) = H{p, e) as a logarithm of a minimal quantity 
N{V, p, e) = N{V, e) = N{p, e) of a balls in the distance p(-, •) S{V, t,e), teV : 

S{V,t,e)=^ {s,seV, p{s,t)<e}, 

which cover the set V : 

N = mm{M : 3{t,}, i = 1,2, , M, t, e V, V C ufL,S{V, t^, e)}, 
and we denote also 

Wn/ ^ A — Ir^n- AT- .Q(+^ A — ^ Q/'T/ +„ A 



H{p,e)''^' H{T,p,e). (3.18) 

It follows from Hausdorf 's theorem conversely that Ve > ^ H{V, p, e) < oo iff the 
metric space {V, p) is precompact set, i.e. is the bounded set with compact closure. 

We quote now some results from [13], [14], [15] about the non - asymptotic exponential 
estimations for Q{u) = Q(T,u) as u >> 1. Define for any value S G (0, 1) and arbitrary 
subset V of the space : V G Q and some semi - distance •) on the set T the 
following function: 

oo 

G{V, p, 5) = G{p, S)^J:S^-' ■ H{V, p, O ■ {1 - S). (3.19) 

m=l 

We define formally G{6) = +oo for the values 6 > Sq. 

In the case when V = U{v) and p{t,s) = ^^(t,^), i.e. when p is the natural semi - 
distance, we will write for brevity G{5) = G{U{v), d^, 5). 

If 



35o e (0, 1), V5 e (0, 5q) G{5) < oo, (3.20) 

then 

Q{T,u)<V{T,S,u), y(T,5,«) =^exp(G'(5) - 0*(«(1 - 5))), (3.21) 

or equally 

Q{T,u)<^mi^V{T,S,u). (3.22) 

If for example 

yS e (0, 1/e] =^ G{T, d^, S) < Ho + k\ log(5|, Hq, k = const < oo, 
then we get denoting 

7r(u) — u (f)*^ {u) 
for the values u for which 7r(it) > 2k : 

Q{T,u) < exp(i/o) K-" (7r(M))'^ exp (-</>*(«)) , (3.23) 

and the last estimation (3.23) is exact in the main (exponential) term exp((—(f)*(u)) . 
More exactly, in many practical cases the following inequality holds: 

Ve e (0, 3/4) 3K > 0, W> K ^ n{u) < exp( - 0*(e m)); (3.24) 
and we conclude hence foi u > K = K{e) by virtue of convexity of a function (f)*{x) : 

Q{T,u)< G,{K,<f>{-)) exp(-0*((l -e)u)), 

and conversely there exists a r.v. ( with unit norm in the space B{(f)) : C e S(0), | |C| | = 1, 
for which 



The value k is called the metric dimension of the set T relative the distance d = d^{- , ■) . 
Note that if 

T = U-=iT(m) 

is some measurable partition R — {T(m)} of the parametrical set T, then 



and hence 



Q(T,u)< Y,Q{T,^,u) 

m=l 



Q{T,u)< inf Vg(T^,«). 

R={T(m)} 



Estimating the right side term by means of the inequality (3.19), we get: 



Q(T,u)< inf 

' ^ R={T{m)} 



E inf EmH,<^M,^) 

d(m)e(0,l) -I 
.m=l \ / V ' ^ 



(3.24) 



The last assertion is some simplification of the Majorizing Measures, or Generic Chain- 
ing Method (see [5], [18] - [21], [2], [11] etc). 

Further we will use as a rule the partition R of the set U{v) of a view 

R = U^=i{e : e e U{v), r{e, Oo) e [ A{k) V, A{k +l)v]}. (3.25) 
4. Main results. 



A. Compact parametrical set. 

The compactness means by definition that the function 9 ^ r{9,9o),9 E Q is bounded. 
Since as a rule the parametric set © is a closed subset in Euclidean finite - dimensional 
space and r(-, •) is ordinary distance, this definition coincides with usually definition of 

the compact sets. 

Note that in this case only finite numbers of the sets {A{k)} are non - empty. We 
can suppose in this subsection for simplicity Ui{v) = U{v) and therefore 0(A) = x(A). 

Let the function = 0(A) be defined as in (2.5a) or equally (in the considered case) 
as in (2.5). Recall that 

sup \\L\U)\\B{x)^l- 
Introduce the so - called natural semi - distance on the set U {v) as follows: 
d^d^^d{9^,92)^\\L^{U^) - L^iUMBix)- 

l|log[/(e,^i)//(e,^2)] - [M^i) - M^2)]||S(x). (4.1) 

It follows immediately from (3.18) (or equally from (3.19)) the following result. 
Theorem 4.1. a.. If there exists 5q — const e (0, 1) such that \/5 e (0, 1) =^ 



G{U{v),d^,6) := ^ 5" - 1 H{U{v),d^,n (1 - 6) < oo, (4.2), 



m=l 



then V5 e (0, So] 



W{v)<exp[G{Uiv),d^,5) - x*((l - S)Y{v))]. (4.3) 

Let us offer the more convenient for apphcation form. Define for U (Z U{v), arbitrary 
function z/ G and any semi - distance p = p{Oi, 62) on the set U the following function 
(if it is finite) 

•^^{U, p, y) =^ inf exp \G{U, p, S) - iy*{{l - S)y)]. (4.4) 

Theorem 4.1. Under the conditions of the theorem (4.1. a) the following estimate is 
true: 

W(v)<^f^(U(v),d^,Y{v)). (4.5) 

B. Non - compact set. 

In this case we need to use the main idea of the so - called generic chaining , or 
majorizing measure method (3.22), (see [5], [18] - [21], [2], [11] etc), which used in 
particular the partition U{v) — [JkUk{v). 

Let us denote for the partition R — {Uk{v)}, U{v) — [JkUk{v) 

Tk^Tkiv)^ sup \\L\i,e)\\B{<P), 
e&Uk{v) 

Y,{v)= \ni h{e), 
eeUkiv) 

and introduce the following distance dk on the set Uk — Uk{v) : 

dk{ei'\ei'^)^\\L%^,9['^) - L%^,9i'^)\\B{cj>),9i'\ ei'^eUkiv). 
Theorem 4.2. We have for arbitrary partition it! 

W{v) < f: (Uk{v), . (4.6) 



k=l 

Notice that 



Tfc('y)' Tk{v) 



H{V, p/K, e)^ H{V,p,K -e), const > 0. (4.7) . 

Proof of the Theorem 4.2. We use the inequality (2.2): W{v) < J2kWk{v). Let us 
estimate each summand Wk{v) : 

Wk{v)^p( sup L{^,e)>o] = 

\eeUkiv) ) 



p sup [L'{^,e) - h{e)]>o] < 



P ( sup L°(^, 6) > Y,{v)] = P ( sup > . (4.8) 

\0eUk{v) J \0eUk{v) rk[v) rk[v) J 



The random field 



is normed in the B{(j)) sense: 



sup 11^^(^)115(0) = !. 



Further, 



\U0^'^) - e.(^f)l|5(0) = 4(^f\^f). 



Using the inequahty (3.22) for the probability Wh{v) and summing over k, we arrive 
to the estimation (4.6). 

5. The reguleir, or smooth case. 



A. Non - formal introduction. Restrictions. Conditions. 

In this section we consider the case when the set © is closed (may be unbounded) 
convex nonempty subset of the Euclidean space i?™, m = 1,2, ... , the density f(x, 9) is 
twice differentiable function on the variable (variables) 6. 

We choose as the deviation function hereafter r{9, Oq) the ordinary Euclidean distance 

r(^i,^2) = - 92,9, - ^2)'= 1^1 -92\. 

The function is in this section the natural, i.e. 0(A) = 0o(-^)- 
We have formally as ^ — > ^0, denoting V / = grad f — df /d9 : 



h{9)^ [ f{x,9o) X 



, f f{x,9o)+Vf{x,9o){9 - 9o) + 0.5V^ f{x,9o){9-9o,9 - 9o) \ 

Cr{9,9of^C\9 - 9o\^ C = C(/(-, •), ^0). 
It is reasonable to assume that 

h{9) X C 1^ - 9o\\ (5.1) 



B. Main result of this section. 



Theorem 5.1. 



We impose on the our statistical structure the following conditions. 

A. Let the function 0(A) = (poi^X) satisfied the condition (2.6) on the set T = U{1). 

B. Assume that the condition (5.1) is satisfied. 

C. Suppose there exists a constant C > 1 such that for each constant K > 1 the 
foUowing inequahty holds: 



sup 'If ^^'„7 <C-K; (5.2) 
e:v<\e - eo\<K v It' — t'ol 

Then there exists a constant C — C{f{-, •), m, ^o) G (0, oo) such that for all the values 
v>l 

W{v) <ex.p{-(l)*{C -v)). (5.3) 

Proof. 

1. We intend to use the resuh of the theorem 4.2. First of aU we choose the partition 
of a view: R = U^f A{k) v, A{k + 1) v], where A(k) ^ k, A; = 1, 2, ... . 

2. Prom the conditions B, or equaUy the condition (5.1) and the condition C foUows 
that: 

Tk{v) <C2{k + l)v (5.4) 

and 

Yk Yj,{v) > CsAy. (5.5) 

3. Since the function </)(■) = 0o(") satisfies the condition A , we can estimate the 
natural distance as follows: 

C4 \ef^ - e'i\. (5.6) 

Since the layer [/^ is bounded in the Euchdean metric, we conclude from (5.6) that 

H{Uk{v),dk/Tk,5) <CQ{L,m) +m \\og5\. (5.7) 

On the other words, in the considered regular case k = m. 

Therefore, all the conditions of theorem 4.2 are satisfied, and we obtain from the 
inequality (4.6): W{v) < Wo{v), where 

00 

Wo{v) =^ E exp {-(l>*{k' v'/iCr {k + 1) v))) < 

k=l 



E exp (-0*(C8 k v)) < exp (-0*(C9 v)) (5.8) 

k=l 



as long as V >1. 



This completes the proof of theorem 5.1. 



Corollary 5.1. The conclusion of the theorem (5.1), i.e. the inequality (5.3) may be 
rewritten as follows. For all the values v > 

W(v) <mm[l,Wo(v)]. (5.9) 

Note that 1^0(0) = +oo. 
Corollary 5.2. 

We obtain using the asymptotical behavior of the function cp = (p{X), A — > 0+ in the 
bounded interval of the variable v : v & [1, Ci], Ci = const > 1 

W{v) < exp (-C v^) . (5.10) 

Notice that under some additional conditions, see [7], chapter 3, section 3, at v < 1 
the following inequality holds: 

W{v) < cxp (-C2 v^) . 
Therefore, we get under these conditions at v < C3 = const > 

W{v) < exp (-C4 v^) . (5.11) 

Remark 5.1 We conclude in the smooth case, taking the union of inequalities 5.8 and 
5.11 and taking into account the behavior of the function (j)*{X) as A — > 0+ : as in the 
case of the of the function 0(A) : 

0*(A)~A^ Ae[0,C], 

that 

W{v) <ex.p[-(l)*{C v)]. (5.12) 
We obtained the main result of this report. 
6. The case of sample. 

In this section we consider the case when ^ = ^ = i = 1, 2, . . . , n are i., i.d. r.v. 

with the (one - dimensional) density f{x,9), satisfying all the condition of the sections 
1 and 5, ( the smooth case.) 

We keep also all notations for the function /(•,•), for instance the notions 
/i(^),F(i;),0(-),0, R etc. 

We will investigate in this section the non - uniform probability under natural 
norming ^/n : 

Wn{v)^P{V^r{0l,9o)>v), (6.1) 
where On — is the MLE estimation of the unknown parameter 9o on the basis the sample 



or equally 



6 = 9n = argmax TT f{i{i), i 



9 = argmax L(^, 9) = argmin(— L(^, 9)) 



(6.2) 



where the contrast function L{-,-) may be written here as 

n 

L = L(^^ = L(")(e, 9) = ^log [m), 9)/ mi), Oo)] (6.3) 

i=l 

and correspondingly 

Lo = ^ L(-)(e,^) = j2log[fm,0)/fm,0o) - h{9)] (6.3.a) 

i=l 

and find also the upper estimation for the uniform probability 

W{v) =supWniv). (6.4) 

n 

Theorem 6.1 Under the formulated conditions the following estimations are true: 

Wn{v)<e^p{-<f>:{C,v)), (6.5) 



W{v) < exp [-(f)* {Ci v) . 
Proof. Let us denote for brevity 

7y(^,^)=7y(^)=log[/(e(^),^)//(e(^),^o)], 

ri°(i) ^rj%i,9) ^rj{i) - Erj{i) ^ rj{i) - h{9). 
We have using the key inequality for the sample of a volume n : 

Wn{v)<p( sup -^j2v{h0)>o] < 
pf sup ^-^2^,9) - h(9)]>o) < 



(6.6) 



1 



sup 



P sup 



\.eeu{v/^) li=i 
1 



i=l 



(6.7) 



As long as 



we can use for the estimation of the distribution of the r. 



/t{v/^) 



and the difference 

Cn(^l) - UO2) 



1 



i=l 



/T{v/y/E) 



(6.8) 



the definition of the function (f)*{-) and its properties; another approach in the many 
general cases, i.e. when the function 0(-) does not exists, may be investigated by means 
of the Lemma 3.1. 

Using the estimation (5.12), we affirm 

W„(t.)<cxp(-0:[Ct;]). (6.9) 
The second assertion of the theorem 6.1 follows immediately by passing to sup„ . 

Remark 6.1 From the assertion of the theorem 6.1 it may be obtained the estimations 
from integral measures of deviation. For instance, if we choose the loss function /(•) of a 
kind 



then 



3C e (0, 00), sup I Ix/^ r(4, 0)\\B{(l)) = C{f{; •)) < 00. 

n 

As a corollary: for all values p — const e [1, 00) 

sup \^/n r{9n,9)\p < Ci p/4> ^(p) < 00, 

n 

where we used the ordinary notation: for arbitrary r.v. C 



(6.10) 



(6.11) 



icIp = [Eicn 



i/p 



7. Some examples. 



Example 7.1. Spherical unimodal distributions. 



We consider the following first example (and other examples) in order to illustrate the 
precision of the theorems 4.1, 4.2 and 5.1. 

Let q = const > 2, X = R"^, be an usually Lebesgue measure, x e R"^ =^ l^^l = 
(x, x)^/^; R{y),y e [0, 00) be twice continuous differentiable strictly positive: 

inf R{y)>0, inf y'^R{y)>0, 

ye[0,l] ?/G[l,oo) 

slowly varying as y ^ 00 functions such that the function y — > y'^R{y), y > is strictly 



Let us introduce the following density function 

fo{x)^C{q,m,R) exp , 
where C{q, m, R) is a norming constant: 



/ fo{\x\) 



dx — 1. 



We take as a parametric sei Q — X — R^; choose '■— 0, and define the family of a 
densities of a view (shift family): 

f{x,9) = fo{\x e\), eee = R"'. (7.1) 

Recall that the observation (observations) ^ has (have) the density of distribution 

/o(N)- 

It follows from the unimodality of the density function that the MLE of the parameter 
9 coincides with the observation ^ : 

- e- (7.2) 

A. Upper bound. 

It follows after some computations on the basis of the theorem 5.1 that (using the 
classical results from the theory of slowly, or regular varying functions functions) (see 
[16], pp. 41 - 53) that for the function 0(A) = |A|« R{\X\) the Young - Fenchel 
transform has a following asymptotic: as A — > oo =^ 

(|A|« i?(|A|))* ~C \\f/R{\\f- 

where as usually 1/p + 1/g = 1. As long as g > 2, we conclude that p e (1, 2]. 
We obtain on the basis of theorem 5.1: v > 1 => 

W{v) < exp [-C3 yP /R {y^ " ^]) . (7.3) 

B. Low bound. 

We get using the explicit representation (7.2) and passing to the polar coordinates: 
W{v) ^^{\i\> v) ^ C{q,m,R) [ exp (-|x|* i?(|j;|)) dx ^ 

Jx:\x\>v 

roo 

C,{q, m,R) - 1 exp (-y« R{y)) dy > 

Jv 



Clo e-K^i-v"^ R{v)/Cn). (7.4) 

Notice that the upper (7.3) and low bounds (7.4) exponential coincides if for instance 
p — q — 2 and R — const (the Gaussian case). 

Analogously may be considered a more general case of the classical MLE estimations. 

Example 7.2. Smooth sample. 

We suDDOse here that all the conditions of the theorem 6.1. are satisfied. 



It follows from the formula (6.8) that 

Wn{v) < exp (-n (f)*{Ci v/V^)) . (7.5) 

Assume that the variable v belongs to the following zone: for some nonrandom positive 
constant C < oo 

v<C2V^. (7.6) 

(a big zone of great deviations). Substituting into (7.5) and taking into account the 
behavior of the function 0* = (p* (A) we obtain in the considered zone the estimation: 

WM < exp (-C3 v') . (7.7) 

On the other hand, we observe that from the CLT for MLE estimations that for each 
fixed positive value v : 

W{v) > lim^Wniv) > exp (-C v^) . 
Example 7.3. Heavy tails of distributions. 

We consider here the sample of a volume n from the standard one - dimensional 
Cauchy distribution: X ^ R^, ^ e ^0 = 0, 



i + {x - ef 

It is easy to calculate that 



</.(A) X Ci |A| < C2; 

0(A) XC3 |A|, |A| > C2. 

More fine considerations as in the theorem 6.1 based on the exponential and power bounds 
for random fields maximum distribution based on the monograph [13], chapter 3, see also 
[14] show us that 

Wn{v) < exp (-C4 v''),v< C5; 

Wn{v) < C^/V, V > C5. 

Therefore, 

supW^„(^;) <C7/^;, ^;>C8. (7.8) 

n 

On the other hand, 

supW^iv) > W,{v) = P(|e(l)| > ^;) > C^/v, V > Cio, (7.9) 

n 

which coincides with upper bound (7.8) up to multiplicative constant. 



At the same result is true for the symmetric stable distributions with the shift param- 
eter 9 : f{x, 9) — f{x — 9), 9 E E}. In detail, let {/(•, •)} be again the one - dimensional 
shift family of densities with characteristical functions 

/oo 
e'*^ f{x,9) dx = e''^ " 1*1", a e (1,2). 
-oo 

Using at the same arguments we obtain the following bilateral inequality: 

Ci < W{v) < C2 t; > 1. 
Example 7.4. Scale parameter. 

Let here i = 1, 2, . . . , n be a sample from the one - dimensional distribution 

A^(0,^), 9 > 0,X ^ R\9o^ 1. 

The theorem 6.1 gives us the following estimation: 

W{v)<exp{-C v),v>l. (7.10) 
The MLE 9n has an exphcit view: 

n 
i=l 

The distribution of 9 coincides, up to multiplicative constant, with the known 
distribution with n degree of freedom. 

We can see by means of this consideration that 

W{v)>exp{-C v),v>l; (7.11) 

and moreover for all values n 

Wn{v) > Clin) exp (-C2(n) v) , v > Cs{n). 
At the same result is true for exponential distribution, indeed, when 

f{x,9)^9-' exp{-x/9); 

X^Rl, 9>0, 9o^ 1. 

Notice that in this case the value Aq from the definition (2.6) is finite 
Remark 7.1 

Note that the case of the so - called penalized modification of the MLE estima- 
tion (PMLE) may be considered analogously. See for definition and first results in the 
nonasymptotic risk estimations in the PMLE ( [17] ) and reference therein. 



(7.12) 
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